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ABSTRACT 


The skill of three ensemble predietion systems (EPS) is evaluated to focus on tropical 
cyclone (TC) track forecasting over the North Pacific. Probability ellipses are defined to 
represent ensemble spread and encompass 68 % of the ensemble members. The ellipses 
are centered on the ensemble mean forecast position. Forecast reliability is defined as 
whether the verifying position is within the ellipse 68% of the time. A statistical analysis 
of uncertainty in TC track forecasts examines the attributes of reliability and resolution of 
each EPS. The European Center for Medium-Range Forecasts (ECMWF) EPS had the 
highest degree of reliability and resolution. The sizes and shapes of the EPS ellipses 
varied with TC track characteristics. This suggests that EPS-based probability ellipses 
may provide value in identifying uncertainty with respect to likely TC track forecast 
error. 
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I. 


INTRODUCTION 


A. MOTIVATION 

Tropical Cyclones (TCs) are among the most damaging and eostly synoptie-seale 
weather phenomena that threaten military aviation and maritime assets and DoD 
installations along eoastlines and on island chains. In the North Paeifie basin, there are 
35 DoD installations, of which 25 are loeated in the Western Paeifie (WPAC) region. 
Also, the WPAC experienees the highest number of TCs of all oeean basins. Sinee these 
installations and their assigned Areas of Responsibility (AOR) overlap with the region of 
known maximum elimatologieal TC oeeurrenee, there are major fiseal and safety 
eoneerns to be addressed by installation and unit eommanders when the threat of a TC 
exists. 

During the period from 2009-2011, there were 54 named storms in the WPAC. 
Eaeh of these storms had the potential to impaet operations at sea, in the air, and on land. 
Monetary impaets of these storms ean reaeh signifieant levels due to the eosts of moving 
ships and aireraft, evaeuating personnel, and preparing installations for damage 
prevention. Potential losses of property and life eaused by a TC eould be mueh more 
signifieant without proper preparation. It is the responsibility of each installation, or unit 
commander to make a highly informed deeision to mitigate the risk of sustaining 
damages. 

The regional Naval Fleet Oeeanographer, with the support of his staff and AOR 
foreeasting center, use numerieal models, offieial AOR foreeasts, and TC produets 
provided by the Joint Typhoon Warning Center (JTWC) to evaluate risk based on a set of 
fixed parameters and limits. This risk assessment will lead to a reeommendation by the 
Fleet Oceanographer to the Fleet Commander to set conditions of readiness (COR), or 
sortie assets to areas of safety. Destruetive winds, storm surge, and sea heights are the 
most signifieant parameters upon whieh the Fleet Oeeanographer bases his 
reeommendation. However, it is destruetive winds that earry the most weight when 
setting COR eonditions. Guidanee for what exaetly defines the magnitude limit of 
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destructive winds, and the time period in advance for which action must be taken before 
these limits are reached, is stated in the instructions published by the Fleet, installation, 
and unit commanders. 

While the Fleet Oceanographer has many sources from which to acquire TC 
forecast information, the official TC forecast for the WPAC is produced by the JTWC in 
Pearl Harbor, Hawaii. These official operational forecast products include written 
forecast discussions and visual decision aids that provide specific information regarding 
TC formation, position, projected track, and maximum wind speeds. While the offered 
TC forecast consists of specific track and intensity values, the uncertainty about these 
values can vary significantly. Many factors affect the uncertainty about a specific 
forecast. One indication of uncertainty is the degree of variability that may exist in the 
aids available to a forecaster. A wide range of forecast scenarios produced by 
independent numerical forecast models is typically an indication that uncertainty is high. 
While variability may change from forecast to forecast, current operational conveyance 
of uncertainty is based on a seasonally static, isotropic swath about the operational 
forecast. 

One method used by JTWC to provide aid in generating forecasts and products is 
to construct a consensus of multiple independent operational deterministic forecast 
models. Goerss (2000) examined consensus forecasts to show that they typically have 
less error than any one single forecast. However, the issue of uncertainty was not 
addressed until Goerss (2007) used a regression analysis to show that the variability 
among deterministic forecasts could be used to define a measure of forecast uncertainty. 

An alternative method to define uncertainty is to use an Ensemble Prediction 
System (EPS), which uses multiple perturbations of the initial conditions to an 
operational numerical weather prediction model. Based on the results of Goerss (2007), 
it may be expected that on average, the mean of individual ensemble forecasts would be 
more accurate than individual forecasts. Eurthermore, a collection of individual 
ensembles prediction systems could be constructed to define a grand ensemble. Although 
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the ensemble-mean foreeast may be more aeeurate than any one foreeast, eonveyanee of 
uneertainty requires some measure of variability as defined by the individual ensemble 
member. 

In this thesis, three individual EPS from three operational models are evaluated to 
examine the use of dynamie probability ellipses to better define the uneertainty in the 
prediction of a TC track. The relative skill of each ensemble-mean forecast TC tracks 
will be examined. These ensemble-mean forecasts are not compared to operational 
forecasts or deterministic forecasts due to differences in forecast reception relative to 
operational constraints for production of a forecast. Dynamic probability ellipses will be 
used to examine the character of each ensemble prediction system in relation to proper 
conveyance of uncertainty via examination of forecast reliability and resolution. 

The products generated by JTWC, such as the official forecast track, wind speed 
probability swaths, and area of uncertainty, are weighed heavily upon by the Fleet 
Oceanographer when assessing risk. Operational JTWC forecasting methods, official TC 
track forecasts, and the production of the “area of uncertainty” will be further explained 
in Chapter II. The case of Super Typhoon Nanmadol (2011) demonstrates how 
challenging it can be to correctly forecast TC tracks when there is a high degree of 
uncertainty contained in numerical model forecast aids. This uncertainty makes it 
difficult for forecasters to determine forecast positions and track directions that are 
needed to better inform the Fleet Oceanographer and Commanders in their decision 
process. In the case of Super Typhoon Nanmadol, it was originally forecasted to impact 
Kadena Air Base, Okinawa, and possibly move to mainland Japan. It is probable that this 
gave base commanders and the 7th Fleet Oceanographer reasons to be concerned. 
Comparison of the official JTWC track forecasts (Figure 1) to the actual track, over the 
duration of the typhoon, conveys the difficulty in forecasting the motion of the storm. A 
brief synopsis of the storm behavior and effects by JTWC was entered into the Annual 
Tropical Cyclone Report (JTWC 2012a): 

Super Typhoon Nanmadol (14W) formed within the monsoon trough east 
of the Philippines and began tracking west-northwestward toward Fuzon 
in a complex steering environment dominated by a subtropical ridge to the 
north and east. It then took a poleward turn around the steering ridge, 
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rapidly intensifying to reach super typhoon status, and clipped the 
northeast tip of Luzon and then moved across the southern coast of 
Taiwan before dissipating in the Taiwan Strait, just prior to making 
landfall in China’s Fujian Province. The cyclone reportedly caused at least 
35 deaths and $34.5M damage in the Philippines, at least 1 death and 
$500M damage in Taiwan, and 2 deaths and $48.5M damage in China. 

While Nanmadol intensified to an estimated maximum wind speed of 140 
kt and caused significant loss of life and property damage, the cyclone is 
also noteworthy from a forecaster’s perspective as the numerical models 
and JTWC track forecasts depiction of an erroneous northeastward turn 
well to the east of the area eventually impacted by the cyclone. This 
tendency in both the model and subjective forecasts began during the 
cyclone development stage in the Philippine Sea and lasted well into its 
mature stage in the Luzon Strait. (JTWC 2012a) 
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Figure 1. The JTWC track forecasts (squares) and best-track (tropical cyclone symbol) 

for STY 14W (From; JTWC 2012a) 
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B. OBJECTIVE 

Operational TC track forecasts provided by fleet weather centers require 
forecasters to assimilate guidance from operational numerical forecast models. As a 
result, the official forecast is often based on a subjective consensus of model guidance. 
The consensus mean of the deterministic models is often more skillful than individual 
operational forecasting systems during periods of increased uncertainty (Goerss 2007). 
An example of large forecast uncertainty is when TC track predictions have a large 
degree of variability among successive forecast integration. Several methods have been 
proposed to reduce uncertainty. The goal of this research is to analyze three operational 
EPS as an aid in identification of uncertainty in TC track forecasts. This will be 
accomplished by comparing the statistical characteristic of each EPS. 

Recently, the ability of European Center for Medium-Range Weather Prediction 
(ECMWE) to predict that a TC would fall within a mean probability circle was studied by 
Majumdar and Einocchio (2009). It was found that the ECMWE ensemble mean (EEM) 
performed comparably to operational consensus techniques based on the mean of 
deterministic forecasts. Additionally, the study concluded that the EEM probability 
circles often did not contain the best-track position during periods of recurvature. Skill of 
forecasts defined by the EEM notably decreased in both EEM and in consensus-mean 
forecasts in the North Pacific during recurvature. The reason was primarily attributed to 
the fact that recurving TCs are often outliers of the ensemble and missed by the mean 
probability circles. 

The operational weather prediction systems to be used in this study include the 
ECMWE, the United Kingdom Meteorological Office (UKMO) Unified Model, and the 
Japan Meteorological Agency (JMA) Global Spectral Model (GSM). While examining 
ensemble-based forecasts of TC tracks over the Atlantic Ocean, Nixon (2012) found that 
the ECMWE provided reliable forecasts in that the best-track positions occurred in the 
68% probability ellipse of ensemble forecasts with a frequency of approximately 68%. 
The GES and UKMO ensembles were found to be less reliable than the ECMWE. This 
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study will provide additional insight into the utility of EPS in providing measures of 
uneertainty of TC track predictions as no studies of this nature have been applied to the 
WPAC region. 

The primary thrust of this research is to develop a spatial and temporal 
representation of the multiple single-model ensembles and demonstrate their utility in TC 
track prediction. Forecast tracks for the year 2009-2011 will be analyzed. These data 
sets will be obtained from the THORPEX Interactive Grand Global Ensemble (TIGGE), 
which will further be defined in Chapter III. It is expected that an appropriate 
representation of ensemble-based forecasts will provide added guidance that will reduce 
the forecast variability and assist in quantifying uncertainty in sortie and COR-setting 
decisions. 

Background material is provided in Chapter II. The methodology used is 
described in Chapter III. The results are presented in Chapter IV and a case study and 
conclusions are given in Chapter V. 
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II. BACKGROUND 


A. JOINT TYPHOON WARNING CENTER OPERATIONAL METHODS TO 

DEFINE TROPICAL CYCLONE TRACK FORECAST UNCERTAINTY 

Operational TC track forecasts can be represented by three depictions of forecast 
uncertainty. Each depiction produced represents an increase in statistical representation 
of uncertainty. The operational track forecast is depicted in an official warning graphic 
(Figure 2) that defines the path of the storm center and an estimate of forecast uncertainty 
that is only based on historical errors. A wind speed probability swath (Figure 3) is 
defined based on a statistical sample of potential track errors by taking a random sample 
from the distribution of official track errors over the previous five year sample. Finally, a 
spread of deterministic models (Figure 4) is used to define the consensus that is used in 
conjunction with the variability among deterministic models to define forecast 
uncertainty. 

I. Official Forecast Track with Area of Uncertainty 

The JTWC predicted TC tracks are displayed on a two-dimensional forecast 
watch/warning graphic every six hours. The official forecast track is shown on this 
graphic as a pink line that defines the predicted track of the center of the TC through 120 
h (Figure 2). The area of uncertainty is the shaded area around the forecast track, which is 
calculated by adding the JTWC 5-year running mean forecast track error to the forecast 
34 knot wind radii at each forecast time. Since JTWC does not forecast wind radii at the 
96- and 120-h, the area of uncertainty is calculated by adding the 72-hour 34 knot radii to 
the forecast track error at those times (JTWC 2012b). Thus, the shading highlights the 
area that may be affected by wind speeds exceeding 34 knots for a given JTWC forecast 
based on historical track forecast errors. However, this calculation does not account for 
uncertainty in the track forecast that may be based on an individual scenario or variation 
of numerical aids that are used as forecast guidance at a particular forecast time (JTWC 
2012b). 
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Figure 2. The JTWC Warning Graphie Legend. The solid black TC symbol defines the 
location of a TC, with winds greater than 64 knots. The last solid black TC 
symbol is the current position, and the pink cyclone symbols are forecasted 
positions. The pink circle, around the pink cyclone symbol, is the 64 knot wind 
radii. The shaded area of uncertainty is the 34 knot wind radii plus the average 
forecast track error (From; JTWC 2012b) 

2. Wind Speed Probability Swath 

Since 2006, the cumulative wind speed probability swath has been used as a tool 
for the Fleet Oceanographer and Commanders to better mitigate the risk to safety and 
damages due to winds. The JTWC produces a graphical display of the wind speed 
probability swath (Figure 3). The graphic can be viewed as 34, 50, or 64 kt cumulative 
probability wind speed swaths. It is produced every 12 h, over a 120-h forecast period. It 
is created by using a Monte Carlo Probability (MCP) model to produce 1000 TC track 
realizations (DeMaria et al. 2009). A single track realization is produced by taking a 
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random sample from the distribution of offieial traek errors over the previous five year 
sample (DeMaria et al. 2009). Ail traek error realizations are added to the offieial 
deterministic forecast tracks and then assigned an intensity and wind structure based on a 
wind profile model. A linear model is applied to account for serial correlation, track, and 
intensity dependency (DeMaria et al. 2009). The two major limitations of this product 
are that the sampling distributions do not account for any background flow dependences 
and sampling distributions are static for an entire TC season since they are based on the 
previous five hurricane seasons (Nixon 2012). 

50_kt_cujiiulative_ vndjprblty (%) 120HR FCST ISSUED 2011082400 



Figure 3. Wind speed probability graphic that depicts the likelihood of 50-kt winds will 
occur during the next 120 h issued 24 August 2011. Legend at the top of the box 
explains color scale representing the probabilities of 50-kt winds. The last black 
tropical cyclone symbol represents the position of the storm when the graphic was 

created (From; JTWC 2012c) 


3. The Goerss Predicted Consensus Error (GPCE) 

The GPCE is used as an aid in determining TC forecast tracks over the WPAC by 
predicting track error based on the spread of deterministic models that are used to define 
the consensus, which was found to be positively correlated with consensus model TC- 
track forecast error (Goerss 2007). The GPCE provides forecasters with a circle based 
on 70% confidence (Eigure 4) that the analyzed TC will be located around a point defined 
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by the consensus of deterministic operational numerical models, which is labeled 
consensus version U (CONU). The CONU is computed when track forecasts from at 
least two of the following five models are available (Goerss 2007): Geophysical Fluid 
Dynamics Laboratory (GFDL) Interpolated (GFDI), Global Forecast System Aviation 
Interpolated (AVNI), Navy Operational Global Atmospheric Prediction System 
(NOGAPS) Interpolated (NGPI), the Navy GFDL Interpolated (GFNI), and the United 
Kingdom Met (UKMET) Office Global Model Interpolated (UKMI). Interpolated 
models are defined by moving the model forecast position in space to account for time 
offset between the model initialization time and forecast time. The GPCE technique is 
based on the assumption of an isotropic error distribution around the consensus mean. 
Based on regression analysis between the spread of the deterministic model and forecast 
accuracy, Goerss (2007) defined the isotropic circle size can be defined such that the 
probability of best-tracks is within the circle is 70%. 



80W 78W 7$W 74W TOW SOW 66W 64W 62W 60W SOW 


Eigure 4. Predicted 70% confidence radius (solid circle) of the 120-h CONU forecast 
for Hurricane Isabel on 0000 UTC 13 September 2003. The individual model 
tracks used to create the CONU consensus model are shown. Notice the GPCE 
circle is much smaller than the 120-h radius (dotted circle) used by the NHC 
potential 5-day track area graphic (Erom: Goerss 2007) 
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Goerss Predicted Consensus Error Along- and Across-track (GPCE- 
X) 


The GPCE-X is a modification to GPCE that removes the requirement for an 
isotropic distribution about the consensus mean (Hansen et al. 2010). In GPCE-X 
anisotropic ellipses are defined based on a partitioning of the forecast track error into 
components that are along-track and cross-track (Eigure 5). The ellipse is defined to 
contain 70% of the predicted positions of the members contained in the consensus of 
deterministic models (CONU). 

The XTE defines the portion of the predicted error in the consensus track that is to 
the left or right of the verifying position. The ATE defines the portion of the predicted 
error of the consensus track that is ahead and behind the consensus mean position. The 
GPCE-X improves upon the sharpness of the predicted area by considering along and 
across-track spread, which reduces the area of the ellipse (Pearman 2011). 



Eigure 5. Schematic definition of forecast-track error (ETE), cross-track error (XTE), 

and along-track error (ATE) (Erom: JTWC 2012a) 

B. ENSEMBLE PREDICTION SYSTEMS (EPS) 

Eor analysis of TC tracks, there are a variety of EPS from which to choose. 
Pearman (2011) and Nixon (2012) focused their studies on the Atlantic basin, and 
provided thorough descriptions of ECMWE, UKMET, and National Centers for 
Environmental Prediction (NCEP) EPS in their research. 
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In this study, the addition of JMA/GSM data is important and appropriate due to 
the regional use of the JMA foreeasts. The NCEP ensemble will not be used in this thesis 
due to the laek of ensemble traek foreeasts in the JTWC data arehive. 

The ECMWF, UKMET, and JMA employ different methods to ereate the 
perturbations needed to generate ensemble members. The ECMWF and JMA both ereate 
initial perturbations based on the singular vector method, and the perturbations are added 
to a control analysis. The singular vectors with large singular values represent fast¬ 
growing perturbations over a prescribed time interval under the assumption that the 
perturbations grow linearly (Eorenz 1965). The fast-growing perturbations are 
considered to be responsible for large forecast uncertainty at the optimization time 
leading to sufficient dispersion in the most uncertain directions (Yamaguchi and 
Majumdar 2010). The UKMET uses an Ensemble Transform Kalman Filter (ETKF) to 
create initial perturbations, which are then added to a control analysis. An ETKF 
provides estimates of the true state of the atmosphere that is based upon an optimized 
blend of short-term forecasts and current observations (Bowler et al. 2008). The 
optimization depends on statistical characteristics defined by the ensemble members 
associated with the data assimilation. 

In general, the purpose of ensemble model forecasting is to define a flow- 
dependent representation of model forecast uncertainty that is based on the overall 
chaotic nature of the atmosphere and its representation by the governing equations of 
motion (Eorenz 1963). The first two sections provide a description of the EPS from 
ECMWF and UKMET based on descriptions of Pearman (2011) and Nixon (2012). The 
third section provides a description of the JMA/GSM EPS. 

1. European Center for Medium Range Weather Forecasts (ECMWF) 

The ECMWF EPS has a horizontal resolution of 32 km. There are 50 ensemble 
members and one control member for a total of 51 ensemble members. The control 
member is defined as the deterministic forecast system with no perturbation except 
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integrated at eoarser resolution than the operational deterministie model. The EPS 
foreeasts are initialized every 12 h at 0000 UTC and 1200 UTC. The output foreeasts 
extend out to 384 h at an interval of 12 h. 

The 50 perturbations are ereated by a eombination of three methods: (1) singular 
vector (SV) technique; (2) using differences between the members of an ensemble of data 
assimilation schemes (EDA); and (3) using two different stochastic perturbation 
techniques (ECMWE 2012). The SVs are selected based on the greatest linear growth 
rate in total energy over a 48-h time period for a fixed set of norms, assumptions, and 
spatial targets (ECMWE 2012). Using differences between the members of ED As is 
accomplished by utilizing a set of 6-hour forecasts starting from ten different analyses 
that differ by means of small variations to the observations and the stochastic physics 
(ECMWE 2012). The two different stochastic perturbation techniques are stochastic 
physics and stochastic backscatter. In stochastic physics, the tendencies in the physical 
parameterization schemes are randomly perturbed. In the stochastic backscatter model 
perturbations are added to the vorticity tendencies to replace the kinetic energy damped 
in parameterization that account for unresolved scales (ECMWE 2012). Each method 
yields a different set of perturbations over the Northern and Southern Hemispheres, and 
the tropics. These perturbations are combined linearly, multiplied by coefficients 
randomly sampled from a Gaussian distribution, and then mirrored by reversing the signs 
of the 25 members created to yield a total of 50 global perturbations (ECMWE 2012). 

2. United Kingdom Meteorological Office (UKMO) 

The UKMO EPS is called the Met Office Global and Regional Ensemble 
Prediction System (MOGREPS) and has horizontal resolution of 60 km and consists of 
23 ensemble members constructed from 22 perturbation members created by the use of 
an ETKE, and one control. The MOGREPS is initialized every 6 h at 0000 UTC, 0600 
UTC, 1200 UTC, and 1800 UTC. The output forecasts are available to 144 h at an 
interval of 12 h. 
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3. Japan Meteorological Agency Global Spectral Model (JMA/GSM) 

The JMA EPS is based on the GSM with a spatial resolution of 20 km and has 51 
members defined by 50 ensemble members and one control. The EPS is initialized every 
12 hours at 0000 EITC and 1200 EITC. The output forecasts are available every 6 h out to 
216 h. The JMA EPS is similar in construct to the ECMWE EPS in that the SV technique 
is used to define the initial perturbation (ECMWE 2012). 
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III. METHODOLOGY 


A. DATA 

1. Data Source 

The data for this study include all forecast and actual TC positions for the WPAC 
from 2009-2011. A total of 54 named TCs formed over this period, of which 20 were 
tropical storms (34 to 63 kt winds), 24 were typhoons (64 to 130 kt winds), and 10 were 
super typhoons (greater than 130 kt). The total number of storms in each year was below 
the climatological mean, which is 31 TCs per year. The most significantly below average 
year was 2010 with only 14 named storms. Data for all three years were available and 
used to determine the statistics and tendencies of the EPS forecasts. 

The 2009 typhoon season (Figures 6a and 6b) was the most active of the three 
years, with 22 named storms consisting of seven tropical storms, ten typhoons, and live 
super typhoons. There were 13 typhoons that made landfall, or came close to island 
chains in various regions; nine in the South China Sea region over the Philippines and 
Southeast Asia mainland, two near Guam, and two near Okinawa. The 2009 JTWC 
Annual Tropical Cyclone Report best summarizes the major storms for the season: 

Official and media reports indicated that Typhoon 09W (Morakot) was the 
most destructive cyclone in the western North Pacific in 2009, with 
significant damage reported on Taiwan and Fujian, Zhejiang and Jiangsu 
provinces, China as well as approximately 650+ fatalities reported mainly 
due to landslides from extremely heavy rainfall. STY 20W (Melor) was 
the only system to make landfall near DoD installations in 2009. This 
cyclone formed to the east of Guam, crossed the northern Marianas Islands 
and attained super typhoon intensity to the southeast of Okinawa Island. 

STY 20W subsequently went ashore about 150 miles southwest of several 
DoD installations in the Kanto Plain as a minimal typhoon. (JTWC 2012a) 
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Figure 6. Tropical cyclones that occurred during (a) April through early September 

2009, and (b) September through December 2009 (From: JTWC 2012a) 


The 2010 typhoon season (Figures 7a and 7b) was the least active of the three 
years, with 14 named storms that included five tropical storms, eight typhoons, and one 
super typhoon. There were 11 typhoons that made landfall, or came close to island 
chains in various regions; seven in the South China Sea region over the Philippines and 
Southeast Asia mainland, three near Okinawa with one TC that continued north to make 
landfall over the Korean Peninsula. The 2010 JTWC Annual Tropical Cyclone Report 
best summarizes the major storms for the season: 

Super Typhoon Megi (15W) was the only cyclone to reach super typhoon 
intensity. Typhoon Kompasu (08W) was operationally significant because 
it made landfall on Okinawa as a strong typhoon and on South Korea as a 
minimal typhoon. (JTWC 2012a) 
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Figure 7. Tropical cyclones that occurred during (a) April through early September 
2010, and (b) September through December 2010 (From; JTWC 2012a) 


The 2011 typhoon season (Figures 8a and 8b) had 18 named storms consisting of 
eight tropical storms, six typhoons, and four super typhoons. There were nine typhoons 
that made landfall, or came close to island chains in various regions; four in the South 
China Sea region over the Philippines and Southeast Asia mainland, two over southern 
Japan, and three near Okinawa. The 2011 JTWC Annual Tropical Cyclone Report best 
summarizes the major storms for the season: 
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The TC formation region was displaced north and west again in 2011, a 
characteristic common during La Nina conditions. Several of these early 
to mid-season forming TCs exhibited S shaped, looping, or generally 
erratic tracks, with numerous passages near or over Okinawa. In fact. 
Super Typhoon Songda (04W) passed just west of Kadena Air Base and 
destroyed the WSR-88D Doppler Weather Radar. (JTWC 2012a) 



b) 



Figure 8. Tropical cyclones that occurred during (a) April through early September 
2011, and (b) September through December 2011 (From: JTWC 2012a) 

2. Data Format 

The data used in this study were produced by three operational EPS that are 
contained in the TIGGE (THORPEX Interactive Grand Global Ensemble) database 
online at http://tigge.ucar.edu/home/home.htm . The standard data format is in Cyclone 
XML (CXML), which is a descriptive, human-legible format used to define forecast track 
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positions for all global EPS (Pearman 2011). The CXML format is defined sueh that it is 
comprised of data from multiple simultaneous cyclone observations, analyses, 
deterministic and ensemble numerical model forecasts (AGBOM 2012). Tropical 
cyclone forecasts are placed in the TIGGE CXML data archive for all storms that are at 
least tropical storm intensity. The storm positions are defined for each EPS based on 
individual definitions of a low level circulation center. 

Best-track data are defined by JTWC, and shared online at 
http://www.usno.navv.mil/NOOC/nmfc-ph/RSS/itwc/best tracks/. The best-track storm 
position and intensity are defined by post-season analysis, which includes surface 
observations, satellite images, aircraft reports, and radar images. A best-track entry 
exists for every six hour period (0000 UTC, 0600 UTC, 1200 UTC, 1800 UTC) over the 
lifetime of the storm (Nixon 2011). 

3. Data Homogeneity 

The forecast track datasets were compiled and analyzed for individual EPSs, the 
WPAC region, and specific sub-regions (Eigure 9). Due to model differences, not all 
models will contain forecasts of each storm at every time. Some models may form a TC 
later or earlier than others. Therefore, a homogeneous dataset is defined such that only 
cases that have forecasts for all three EPS are included. In addition, any TC that did not 
persist longer than 48 hours was removed from the dataset. An additional homogeneity 
constraint was applied for the GE, GPCE and GPCE-X forecasts. 

4. Region and Sub-regions 

The database used in this study was compiled specifically for the WPAC region, 

and then divided into specific sub-regions to examine detailed variabilities in forecast 

accuracy and uncertainty. The Monsoon Trough Region (MTR) defines the mean position 

of the WPAC monsoon that provides favorable low-level, large-scale cyclonic shear in 

which convective disturbances form. Most of the TCs that form in the MTR move 

westward to the South China Sea Region (SCSR) or recurve poleward. Recurvature 

typically occurs when the subtropical ridge is weakened and the storm moves 

northwestward due to the weakened easterly winds on the equatorward side of the 
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subtropical ridge. The SCSR is used to capture TCs that move westward under influence 
of the subtropical ridge. The Recurve Region (RCR) defines the latitudinal region where 
TCs normally recurve. Following recurvature, TCs move into the Mid-Latitude Region 
(MLR) and typically move westward under the influence of mid-latitude westerlies. The 
number of TC passing through each region in Figure 9 is defined in Table 1. 



Figure 9. Geographie sub-regions of the WPAC region used to group Ensemble 

Prediction System (EPS) forecast track data 


Table 1. Total number of Tropical Cyclones included in this thesis for the WPAC 

and each sub-region 


Year 

Pacific 
Basin TCs 

Monsoon 
Trough 
Region TCs 

South 
China 
Sea TCs 

Recurve 

Region 

TCs 

Mid- 

Eatitude 

Region 

TCs 

2009 

22 

9 

12 

11 

4 

2010 

14 

5 

11 

6 

5 

2011 

18 

7 

6 

11 

9 

TOTAL: 

54 

21 

29 

28 

18 


20 


















5. Developing the EPS Ellipse 

As explained in Chapter II (Figure 2), the ereation of JTWC forecast track and 
area of uncertainty is one of a number of methods used to analyze and display forecast 
uncertainty. The methods have limitations due to the lack of confidence for TC position 
inside of the area of uncertainty. Carr and Elsberry (2000) quantified the relationship of 
the spread in the consensus of deterministic models to forecast track error. Additionally, 
Pearman (2011) concentrated on understanding the relationships and disadvantages 
between the current methods of calculating uncertainty by creating a Grand Ensemble 
(GE) forecast track ellipse and comparing EPS, GE, GPCE, and GPCE-X forecasts. 

Pearman (2011) examined the uncertainty in the GE forecast position as being 
defined in relation to the principal axis of the spatial distribution of EPS members 
centered relative to the GE mean position. The ellipse is defined to contain 68% of the 
GE member forecast track positions, is centered on the GE mean, and is calculated for 
each forecast period in the homogeneous dataset (Pearman 2011). 


The ellipse calculation performed by Pearman (2011) is a critical procedure in 
understanding the ensemble member spatial distribution. This study employs the same 
procedure to create the EPS and GE ellipses. The initial step is to create a 2 x n matrix 
(A) of the latitudes and longitudes of each ensemble member contained in the EPS, at a 
particular forecast time. The variable n is defined as the total number of forecasts present. 
A covariance matrix is created from the matrix of latitudes and longitudes: 


I, = ao^(A^A)-i = 


CTi 




^12 


( 1 ) 


The next step is to calculate eigenvalues ( , X 2 ) and eigenvectors of the covariance 

matrix: 


I2} ^ ^ ^ - V(^1 + ^1)^ “ (2) 

The resulting eigenvalues scale the ellipse along the semi-major and semi-minor axes, 
while the eigenvectors of the covariance matrix define the orientation, which orients of 
the ellipse axes to be in the direction of the largest variance. 
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The mean of the latitude and longitude matrix is computed to find the center 
position of the ellipse. To ensure that the ellipse captures 68% of the EPS member 
forecast positions, a Chi-squared distribution is assumed and a Chi-squared scaling 
parameter is applied. 

Based on the Chi-squared scaling, the ellipses were defined such that an ellipse 
captures 68% of the EPS ensemble members at a 95% confidence interval, using a 
homogeneous dataset (Figure 10). The orientation of the semi-major axis is determined 
by the spatial distribution of the EPS members. 



Figure 10. The probability ellipse (red) contains 68% of the ensemble members (red 
dots) and is centered on the ellipse mean position, yl and y2 are the coordinate 
system by which the ensemble members are rotated around. The orientation of 

the ellipse is defined hyc/) 


22 











B. STATISTICAL ANALYSES 


Nixon (2011) addressed the statistical characteristics of ensemble forecasts by 
computing the reliability and resolution of individual EPS relative to the 68% ellipse 
resolution. Reliability of a forecast pertains to the calibration of the forecast, which 
relates the forecast to the observation. For forecasts that are perfectly reliable, the 
observed relative frequency of an observation would be equal to the forecast probability. 
For TC location prediction, we define reliability from 68% threshold, when the 68% 
probability ellipse encompasses the best-track position 68% of the time. The resolution 
of a forecast pertains to the ability of a forecast system to sort observed events into 
groups that are different than each other. If the observed position of a TC is quite 
different in forecasts of recurvature versus straight moving, then the forecast system has 
adequate resolution to identify these two scenarios. If the observed location is the same 
for a forecast of recurvature as for a forecast of straight moving, then the forecast system 
cannot resolve these two types of forecasts. The resolution and reliability of each 
forecast interval will be inferred jointly by calculating both the Probability within Spread 
(PWS) and Mean Area Difference (MAD). 


I. Probability within Spread 


Probability within Spread (PWS) defines the probability that an observed TC is 
within the spread of an EPS as: 


PWS 


^obs^ ?nl 


where k, m are integers (k = 1, 2, 3...), M is the total number of forecasts at a given lead 


time. Sobs is the distance of the observed TC from the EPS mean, and a is the across-track 


spread of the EPS (Nixon 2011). If members are sampled from a normal distribution, a 
PWS value of 0.68 for a spread defined by one standard deviation about the ensemble 
mean would indicate statistical consistency (Buckingham et al. 2010). Furthermore, if 
members are sampled from a normal distribution and the spread is Icr (k = 2 a (k = 2), 

and 3 a {k= 3), PWS values of 0.68, 0.95, and 0.99 would indicate statistical consistency 
(Buckingham et al. 2010). The spread is defined by the standard deviation of the EPS 
members about the ensemble-mean position. PWS calculates a percentage that reflects a 
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spatial representation of best track presence within a cross-track conically bounded area. 
For the purposes of this study, a PWS value of 0.68 for 1 a was utilized to indicate 
statistical consistency. 

2. Ellipse Reliability 

Ellipse reliability is the percentage of time that the best-track analysis position is 
within the EPS ellipse at a particular forecast time. Reliability is defined as: 

^ y’M rO:So^5>(68% . .. 

M ll:Sobs- (68% ellipse)^) ^ 

where the definition of the EPS ellipse is defined as enclosing 68% of the ensemble 
forecast track members, resulting in the expected ellipse reliability to be 68% (Nixon 
2011). The reliability percentages of the EPS that are higher (lower) than 68% will be 
determined to be under (over) dispersive. The ellipse reliability statistical analysis 
method is similar to PWS in that it calculates a percentage of events in which the best- 
track is located in a defined area. However, PWS calculates a percentage that reflects a 
spatial representation of best track location in a cross-track conically bounded area. 
Ellipse reliability calculates a percentage that reflects a temporal representation best-track 
location within a defined ellipse that is defined to encompass a set number of ensemble 
members. Eurthermore, the orientation of the ellipse caries based on flow-dependent 
uncertainty, which the speed is based on standard deviation about the mean will be 
symmetric by definition. 


3. Mean Area Difference (MAD) 


The Mean Area Difference (MAD) is a calculation that compares the area of the 
EPS ellipse, which is the forecast ellipse area, with a control ellipse area. This provides a 


percentage difference in area, or MAD, which is defined as: 


MAD 


Control Ellipsearea~ Forecast Ellipsearea 
Control Ellipsearea 


( 5 ) 


A MAD percentage is positive (negative) if the EPS ellipse area is less (more) than the 
control ellipse area. 
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IV. ANALYSIS AND RESULTS 


A. OVERVIEW 

Each individual EPS is characterized by an ensemble-mean traek foreeast and a 
spread or dispersion about the mean forecast. As defined in Chapter II, the primary 
statistical measures used to evaluate the ensemble-mean forecast are the ETE, XTE, and 
ATE. These measures are analyzed to determine whether any significant biases or 
systematie errors exist in the ensemble-mean forecast tracks. The statistical analyses are 
examined for the entire WPAC basin and for each sub-region. 

B. ENSEMBLE-MEAN TRACK ERRORS 

1. ETE 

The PTE is defined as the absolute distance between the ensemble-mean and the 
TC best-track positions. The PTE indicates whieh EPS forecast tracks are closest on 
average to the actual best-track analysis positions. The spread about the PTE will be 
indicated by vertically-oriented bars that define the standard deviation for eaeh mean 
PTE. The standard deviation is defined as the square root of the average squared 
difference between the ensemble-mean track position and the TC best-track analysis 
position. Because the standard deviation involves squared errors, even one very large 
traek error ean strongly impact the magnitude of the standard deviation in the bar graph. 
Based on the PTE, no directionality information can be inferred with respect to the track 
error characteristies for each EPS. 

Por all forecasts over the entire WPAC (Pigure 11), the ECMWF ensemble-mean 
ETE is smaller than the ETE of the UKMO and JMA ensemble-mean forecasts. 
Additionally, the standard deviation about the ECMWF ETE is smaller than the standard 
deviation about the mean ETE for the UKMO and GFS EPS. 
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2009-2011: WPAC Ensemble Mean FTE 



Figure 11. Average FTE for each EPS over the entire WPAC from 2009-2011. Plus and 
minus one standard deviation about the PTE is represented by a vertically oriented 
line at each 12-h forecast from 0-120 h 

Por the Monsoon Trough Region (MTR) (Pigure 12), the FTE of the ECMWF 
ensemble-mean forecast is also smaller than that of the ensemble-mean forecasts from the 
UKMO and JMA. At forecast ranges between 48-84 h, the ensemble-mean PTE for the 
JMA and UKMO are approximately equal. Por the ECMWP forecast intervals shorter 
than 72 h, the PTE over the MTR (Pigure 12) is approximately equal to the PTE over the 
entire WPAC (Pigure 11). Beyond 72 h, the ECMWF ensemble-mean FTE over the 
MTR is less than that over the entire WPAC. By 120 h, the ECMWF ensemble-mean 
FTE over the MTR is 100 km less than the WPAC average. The UKMO ensemble-mean 
PTE over the MTR is also less than the total WPAC value. The JMA ensemble-mean 
PTE over the MTR is approximately the same as the WPAC average. 
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From 0 to 96 h the FTE standard deviations for ECMWF are noticeably lower in 
the MTR than the values over the entire WPAC and only slightly lower at 108 and 120 h. 
The small EXE and standard deviation of the ECMWE ensemble-mean EXE over the 
MTR is probably due to the relatively straight steering flow over the MTR, which is 
typically less variable than the more complex steering flows over other regions of the 
WPAC. 

2009-2011: WPAC Ensemble Mean FTE - Monsoon Trough Region 



Eor the Recurve Region (RCR) (Eigure 13), there is a noticeable increase in the 
ensemble-mean FTE for all EPS beyond 48 h. On average, the ensemble-mean FTE is 
50-100 km larger in this region than in the total WPAC (Eigure 11). The increase in EXE 
can be attributed to the more variable steering flow that is influenced by mid-latitude 
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flow patterns. Additionally, the loeation of reeurvature is often highly sensitive to the 
interaetion between the mid-latitude flow poleward of the TC and the subtropieal ridge 
equatorward and east of the TC. 

The standard deviation about the ensemble-mean FTE is greater for ECMWF in 
this region as eompared to the WPAC and other sub-regions. Noticeably, UKMO has the 
largest mean ETE but smaller standard deviations when compared to the other sub- 
regions indicating it has a larger ETE bias in the RCR. The JMA ensemble-mean FTE 
errors in this region are comparable to the FTE in other regions and the overall WPAC. 


2009-2011: WPAC Ensemble Mean FTE - Recurvature Region 



Over the South China Sea Region (SCSR) (Figure 14), the ensemble-mean FTE 
for the ECMWF are similar to the values observed over the entire WPAC (Figure 11). At 
forecast ranges from 12-96 h, the UKMO ensemble FTE over the SCSR is less than that 
over the entire WPAC. At forecast ranges between 48-108 h, the average FTE for JMA 
is 50 km larger than that over the WPAC. 
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The standard deviations about the ensemble-mean FTE for JMA from 48-120 h 
are larger eompared to the standard deviations for the same foreeast intervals over the 
entire WPAC. The inereased FTE and larger standard deviations for the JMA eould 
indieate poor foreeasts of flow patterns over the SCSR, whieh eould lead to early 
recurvature of the TCs in the mid-range forecasts. The ElKMO values are lower with 
smaller standard deviations from 12-96 h when compared to that of the WPAC. 
Therefore, the UKMO model forecasts are more accurate over the SCSR. The ECMWF 
FTE over the SCSR are just slightly larger than the FTE over the entire basin. 


2009-2011: WPAC Ensemble Mean FTE - South China Sea Region 

800 


700 
600 
J 500 


§ 400 

LU 

S 300 


■K 200 

(Q 
V 
lU 

" 100 
o 


0 

-100 


-1-1— 

♦ ECMWF 

♦ UKMO 

♦ JMA 

-1 

-1 

-1 

-1 

-1 

-1 

-1 

-1 








■ 

■ 

m 













F- 

► 








■ y 

--1 







> 





1 - 




. j 


✓ • 




P 



■ J 






* 1 3 

• ^ 


















i 

!9 1 

??_li 

n _L 

y_L 

No. ofC 
|4_U 

Bises 

H_^ 

ft_ li 

1!_^ 

?_^ 

^ _fti 


0 12 24 36 48 60 72 84 

Forecast Interval (h) 


96 108 120 


Figure 14. 


As in Figure 11, except for the SCSR region 


The characteristics of ensemble-mean FTE over the Mid-Fatitude Region (figure 
15) are much different than the other regions, which is likely due to the small sample 
sizes and the highly variable flow patterns over the region. Small sample sizes may 
produce results based on one very large track error that can strongly influence the 
forecast-error or impact the magnitude of the standard deviation in the bar graph. The 
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ECMWF ensemble-mean FTE is smaller than that of UKMO and JMA (Figure 15). For 
the UKMO forecast intervals shorter than 84 h, the FTE over the MFR is larger than that 
over the entire WPAC (Figure 11). Beyond 84 h, the UKMO ensemble-mean FTE over 
the MFR is less than that over the WPAC. For ECMWF and JMA forecast intervals 
beyond 72 h, the ensemble-mean FTE over the MFR is larger than that over the entire 
WPAC. 


2009-2011: WPAC Ensemble Mean FTE - Midlatitude Region 



2. ATE 

As defined in Chapter II, a positive (negative) Along-Track Error (ATE) indicates 
that the forecast position is ahead (behind) of the best-track position. The ATE reflects 
errors relative to advection of the TC along the correct path without regard for the flow 
orientation. 
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Over the WPAC, each individual ensemble-mean forecast has an average ATE 
between zero and 100 for each forecast interval. Therefore, the average ensemble forecast 
is ahead of the verifying best-track position (Figure 16). For each EPS, the standard 
deviations of the ensemble-mean ATE progressively increase as the forecast intervals 
increase. 


A drastic increase occurs in standard deviation beyond 84 h. The increase in 
standard deviation is expected due to greater forecast uncertainty with increasing forecast 
interval; however, the magnitude of the standard deviation suggest high variability in 
long range TC forecasts as being ahead or behind the actual speed of movement. As will 
be clear when individual regions are examined, the standard deviation of medium-range 
ATE is likely due to large negative errors over the mid-latitude region (Figure 20). The 
ECMWF has the smallest ensemble-mean ATE and standard deviations about the mean 
ATE, which is consistent with the FTE results. 


2009-2011: WPAC Ensemble Mean ATE 



Figure 16. Average ATE for each of the EPS over the entire WPAC from 2009-2011. A 
plus and minus one standard deviation about the mean ATE is represented by the 
vertical line at each 12-h forecast from 0-120 h 


31 





































Over the MTR (Figure 17), the ensemble-mean ATE for all EPS are also positive 
over all forecast intervals. There does not appear to be a significant timing bias in the 
ensemble-mean forecast over the MTR. However, the most significant difference 
between the MTR and the overall WPAC region is the lack of increased standard 
deviation over the MTR at larger forecast intervals. 

2009-2011: WPAC Ensemble Mean ATE - Monsoon Trough Region 



Over the RCR (Eigure 18), the ensemble-mean ATE for all EPS are slightly more 
positive than over the MTR region. Also, the standard deviations about the mean ATE 
are larger, which reflects the variability associated with the TC recurvature. The 
ECMWE and JMA have negative ensemble-mean ATE over the medium-range forecasts. 
This indicates that ECMWE and JMA may be slower to forecast the recurvature of a TC 
at these forecast intervals. 


32 















































2009-2011: WPAC Ensemble Mean ATE - Recurvature Region 



The largest positive ensemble-mean ATE for all EPS oeeurs over the SCSR 
(Eigure 19). This may be due to inereased variability in TC steering flow and erroneous 
forecasting of recurvature. The mean ATE forecast EPS are very similar such that no one 
EPS seems to be more accurate than any other. 
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2009-2011: WPAC Ensemble Mean ATE - South China Sea Region 



Over the MLR (Figure 20), the ensemble-mean ATE for all EPS beeome very 
negative beyond 48 h. Therefore, the forecasts from all EPS are behind the actual TC 
tracks. This indicates that the EPS are not accurately forecasting the speed of TCs along 
the track as they enter the strong mid-latitude westerlies. Eurthermore, the standard 
deviations about the mean ATE are small, which indicates that the speed bias in the 
ensemble-mean forecasts is quite systematic. 
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2009-2011: WPAC Ensemble Mean ATE - Midlatitude Region 



3. XTE 

The Cross-Track Error (XTE) is the component of the ETE that indicates whether 
the forecast position is left (negative) or right (positive) of the best-track position. The 
XTE can indicate an error in predicting the orientation and speed in the motion of a TC 
relative to background steering flow. 

Eor all EPS at forecast interval less than 72 h, the mean XTE is near zero, which 
indicates no clear bias in the ensemble-mean forecasts. The ECMWE and JMA 
ensemble-means have low average XTE across all forecasts intervals in the WPAC 
(Eigure 21). The standard deviations about the mean XTE are small, which indicates 
consistency in predicting the orientation of the background flow. Beyond 72 h, the mean 
XTE becomes positive for each EPS, which indicates slight bias to the right of the best- 
track. 
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2009-2011: WPAC Ensemble Mean XTE 



Figure 21. Average XTE for each of the EPS of the entire WPAC from 2009-2011. A 
plus and minus one standard deviation about the mean XTE is represented by a 
vertical line at each 12 h forecast from 0-120 h 

Over the MTR (Eigure 22), the ensemble-mean XTE characteristics are similar to 
the entire WPAC. At short forecast ranges, the mean XTE are near zero for all EPS. At 
longer forecast intervals, the mean XTE becomes positive, which indicates the same right 
bias as evident over the entire WPAC (Eigure 21). 
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2009-2011: WPAC Ensemble Mean XTE - Monsoon Trough Region 
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Figure 22. 


As in Figure 21, except for the MTR region. 


Over the RCR (Figure 23), ensemble-mean XTEs become more positive than over 
the entire WPAC (Figure 21). The ensemble-mean XTE for the UKMO is much more 
positive than for JMA and the ECMWE, which are similar. This positive XTE bias 
indicates all three EPS consistently forecast tracks to the right of the best-track. This 
could be due to the EPS over-forecasting the influence of increasing mid-latitude 
westerlies as TCs tracks northward. 
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2009-2011: WPAC Ensemble Mean XTE - Recurvature Region 



Over the SCSR sub-region (Figure 24) ensemble-mean XTE for all EPS are 
smaller than over any other regions. On average, there is no defined bias in the 
ensemble-mean forecast tracks. 
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2009-2011: WPAC Ensemble Mean XTE - South China Sea Region 



Consistent with the ATE over the MLR region (Figure 25), the ensemble-mean 
XTE for all EPS becomes very negative at larger forecast intervals. Therefore, all EPS 
have large bias. This is consistent with the negative ATE errors. As a TC turns eastward 
under the influence of increased mid-latitude westerlies, the storm is forecast to move too 
slow, which would place it to the left (west) of the best-track. 
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2009-2011: WPAC Ensemble Mean XTE - Midlatitude Region 



C. PROBABILITY WITHIN SPREAD 

The Probability within Spread (PWS) is a caleulation that defines the proportion 
of time that the best-track is contained within the spread of the individual ensembles. 
Spread is defined as being one standard deviation to the right or left of the ensemble- 
mean track as defined by the individual ensemble members. An EPS that has a high 
(low) PWS for a particular forecast interval indicates that the individual ensemble 
forecast track members do (not) have enough spread to reflect the track uncertainty. For 
the purposes of this study, a PWS value of 0.68 for 1 a was utilized to indicate statistical 
consistency. 

Throughout the intermediate forecast intervals, each EPS has a PWS that is 
between 0.4 and 0.5 over the WPAC (Figure 26). The PWS is small at the initial time of 
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Probability Within Spread 


the forecast due to small perturbations in the initial conditions. The initial condition 
uncertainty leads to large spread at very short forecast intervals. The PWS increases 
rapidly at 12 h in response to the initial condition uncertainties. 


Figure 26. 


2009-2011: Probability Within Spread 
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2009*2011: Probability Within Spread; WPAC Monsoon Trough Region 2009-2011: Probability Within Spread: WPAC South China Sea Region 




2009-2011: Probability Within Spread; WPAC Recun/e Region 2009-2011: Probability Within Spread: WPAC Midlatitude Region 




Figure 27. The PWS for each EPS over each sub-region of the WPAC for the 2009-2011 

seasons 

Over the MTR (Figure 27a), the PWS was larger than the basin average for all 
EPS. However, over the SCSR (Figure 27b) and the RCR (Figure 27c), the PWS was 
smaller, which indicates that spread was too small with respect to potential recurvature 
tracks. Over the MER (Figure 27d) the PWS is larger than the RCR and SCSR. This is 
due to two reasons. There are relatively few numbers of cases or homogeneous forecasts 
members over the MLR. Also, the spread over the MLR is generally much larger than 
the other regions due to the increased variability in mid-latitude westerlies. Therefore, 
the PWS is high, but the resolution is likely to be low. 

D. ELLIPSE RELIABILITY 

In this thesis, reliability is defined as the percentage of time that the best-track 
analysis position is contained within the EPS ellipse at a particular forecast time As 
discussed in Chapter III, the EPS ellipse is defined as enclosing 68% of the ensemble 
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forecast track members, based on a Chi-squared scaling. The reliability percentages of the 
EPS which are higher (lower) than 68% will be determined to be under (over) dispersive. 

In Figures 28 and 29, the reliabilities of individual EPS ellipses are shown by line 
graphs for each forecast interval. The blue line symbolizes the expected reliability of the 
ellipses at 68%. The values above the forecast interval are the number of EPS forecasts 
that were included. 

The reliability, or ellipse hit rate, for all of EPS is below 68% for the WPAC 
(Figure 28) and all sub-regions (Figure 29). The ECMWF has the highest average 
reliability of the three EPS over the entire WPAC (Figure 28). The distribution of 
reliability over the four sub-regions is similar to that of PWS. That is, the reliability over 
the MTR and SCSR is higher than the reliability over the entire WPAC. Ellipse 
reliability over the RCR is smaller than over the entire WPAC. Although, the PWC over 
the MTR is larger, the reliability is the smallest of all regions. 


2009-2011 WPAC Ellipse Reliability 



Figure 28. The ellipse reliability for each EPS for the entire WPAC 
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2009-2011 WPAC Monsoon Trough Region Ellipse Reliability 


2009-2011 WPAC South China Sea Region Ellipse Reliability 
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Figure 29. The ellipse reliability for eaeh EPS over the eaeh sub-region of the WPAC for 

the 2009-2011 seasons as defined in Figure 28 


E. MEAN AREA DIFFERENCE (MAD) 

Beeause the ensemble-mean foreeasts for the ECMWF generally exhibit the 
smallest FTE, ATE, and XTE, the MAD ealeulation is based on the ellipse area of the 
ECMWF. The MAD value measures the sizes of the UKMO or JMA ellipses relative to 
the ECMWF ellipse. A positive (negative) value indieates that the UKMO or JMA ellipse 
is smaller (larger) in size than the ECMWF ellipse. This would indicate that the UKMO 
or JMA predicts a reduced (higher) level of uncertainty. The values above the forecast 
interval show the number of EPS forecasts included. 

Over all forecast intervals the MAD for the UKMO and JMA with respect to the 
ECMWF is negative (Figure 30) for the entire WPAC. At forecast intervals beyond 60 h, 
the MAD values are consistently near -1.00 for the UKMO and near -2.5 for JMA. 

Therefore, the JMA ellipse sizes are the largest of all three EPS investigated in this thesis. 
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2010-2011 WPAC Ellipse MAD: ECMWF vs (JMA and GFS) EPS 



Figure 30. The Mean Area Difference (MAD) of UKMO or JMA EPS and compared to 
ECMWF across the WPAC. Positive values indicate the EPS ellipses are smaller 
than the ECMWF circle for each forecast interval. The values above the forecast 
interval are the number of EPS forecasts included 

F. SUMMARY 

The objective of examining TC track errors from individual ensemble models is to 
enhance and analyze the attributes and utility of forecast reliability and resolution with 
respect to the ensemble spread as an aid to TC forecasting. Overall, errors were smallest 
for ECMWF. Additionally, the standard deviations about the mean error were smallest 
for ECMWF. ATE over the entire WPAC indicates average ensemble forecasts are ahead 
of the verifying best-track position. The average XTE over the entire WPAC for 
forecasts beyond 72 h indicates a slight bias to the right of the best track. 

In the MTR there is a slight right bias. However, this contains less error 
variability when compared to other regions in which steering flows may be more 
complex steering. 
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In the RCR, it is evident that the variable steering flow and foreeasted TC 
reeurvature is influeneed by mid-latitude flow patterns. The EPS tend to over-foreeast 
the influenee of inereasing mid-latitude westerlies as TCs traek northward. Of note, 
ECMWF and JMA may be slower to foreeast the reeurvature of a TC at these foreeast 
intervals. 

In the SCSR, the inereased FTE and larger standard deviations for the JMA 
ensemble mean eould indieate poor foreeasts of flow patterns over the SCSR leading to 
early reeurvature of the TCs in the mid-range foreeasts. The UKMO model foreeasts are 
the most aeeurate over the SCSR based on FTE analysis. However, there is no defined 
ATE or XTE bias for any EPS. 

In the MER, the eharaeteristies of ensemble-mean FTE are mueh different than 
the other regions, whieh is likely due to the small sample sizes and the highly variable 
flow patterns. The ECMWF ensemble-mean FTE is smaller than that of UKMO and 
JMA. The ATE for all EPS indieates that foreeast positions are behind the aetual TC 
traeks. This indieates that the EPS are not aeeurately foreeasting the speed of TCs along 
the traek as they enter the strong mid-latitude westerlies. Furthermore, the standard 
deviations about the mean ATE are small, whieh indieates an along traek bias, or bias in 
translation of speed. The XTE is eonsistent with the ATE in that all EPS have large bias 
to the right of the traek, whieh is eonsistent with negative ATE. 

The PWS is small and relatively similar for eaeh EPS throughout the WPAC, 
whieh indieates that the probability of the observed traek being within the ensemble 
spread was small. There is no elear distinetion as to whieh EPS had the highest PWS. 

The ellipse reliability for all of EPS is below 68% for the WPAC and all sub- 
regions, whieh indieates under-dispersion. The ECMWF has the highest average 
reliability of the three EPS over the entire WPAC. The distribution of reliability over the 
four sub-regions is similar to that of PWS. That is, the reliability over the MTR and 
SCSR is higher than the reliability over the entire WPAC. 

The negative MAD values for the UKMO and JMA indieate that these EPS result 
in ellipses that are larger in size than the ECMWF ellipse. Thus, the UKMO or JMA may 
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overestimate uncertainty. The increased size of the UKMO and JMA ellipses, as 
compared to the ECMWF, also indicate the decrease of sea maneuverability when relying 
on UKMO and JMA to establish visual aids that convey TC track areas of uncertainty. 

In summary, the statistical analyses which proved most informative were the 
basin and sub-regional track-errors and the MAD. The track-errors conveyed amplifying 
information about which individual EPS performed best over the basin and in specific- 
sub-regions. The UKMO ensemble showed merit in the forecasting-track error in the 
SC SR, while the JMA ensemble did not clearly distinguish itself in any of the sub- 
regions. The PWS and MAD, along with the majority of the track-error analysis, 
conclude that the ECMWF ensemble consistently outperforms the UKMO and JMA 
ensembles over a majority of the WPAC and the sub-regions, when comparing the 
attributes of reliability, resolution, and sharpness of forecasts. 

G. CASE STUDY TYPHOONS NANMADOL AND SONGDA 

Typhoon Nanmadol was chosen in this case study because of the high level of 
uncertainty surrounding its forecast track and the potential to impact Okinawa and 
mainland Japan (Figure 31). Typhoon Sondga was chosen to contrast with Typhoon 
Nanmadol because it has a lower level of uncertainty (Figure 32) and forecasts were 
generally along the best-track. The initial JTWC track forecasts for both Songda and 
Nanmadol were both very close to Okinawa. 
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Figure 31. 



All JTWC track forecasts for Typhoon Nanmadol. The red dashed lines 
correspond to each JTWC forecast track. The best-track positions are in black 

(From: JTWC 2012a) 
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Figure 32. All JTWC track forecasts for Typhoon Songda. The red dashed lines 

correspond to each JTWC forecast track. The best-track positions are in black 

(From: JTWC 2012a) 


1. Typhoon Songda 

On 22 May 2011, Super Typhoon Songda formed east of Palau (Figure 32) and 
rapidly intensified to a peak of 140 kt by 27 May as it recurved east of the Philippines 
and Taiwan (JTWC 2012a). By 28 May, Songda subsequently weakened to 80 kt under 
the influence of increasing vertical wind shear as it passed approximately 40 nautical 
miles to the north-northwest of Okinawa (JTWC 2012a). On 29 May, the cyclone passed 
along the southern coast of Honshu before completing extra-tropical transition and 
accelerating eastward into the central North Pacific as a baroclinic low pressure system 
(JTWC 2012a). 


a. Typhoon Songda 1200 UTC 23 May 2011 

The Typhoon Songda case study begins 1200 UTC 23 May 2011 (Figure 
31). The ECMWF was the only EPS that had significantly smaller along- and cross-track 
components, and a corresponding small spread of ensemble members. This is consistent 
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with the summaries of XTE and ATE in the MTR (Eigure 17 and 22) and RCR (Eigure 
18 and 23). The ensemble members of UKMO also had very little spread in the 
individual forecast members, but the ellipses were larger in both along- and cross-track 
components in the short range when compared to ECMWE. The UKMO forecast spread 
increased over the mid- to long-range forecasts near the location of forecast recurvature. 
The JMA ensemble spread was the greater over all the forecast intervals. The ellipse 
orientation of the JMA EPS in the short-range forecast has a large cross-track component, 
and in the long-range they show a long along-track component. This indicates that JMA 
predicted the greatest uncertainty as to when Songda will begin to recurve. 


Ensemble Forecasts and 68% Ellipses: Typhoon Songda: 2011052312 
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Eigure 33. The TC forecast track ellipses of each EPS for Typhoon Songda 1200 UTC 23 
May 2011. Each ellipse signifies a 12 h forecast interval and is colored to match 
the individual EPS as defined in the legend at the top right. The large dot inside 
each ellipse is the corresponding ensemble-mean forecast position. The best-track 

positions are in black 
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b. Typhoon Songda 1200 UTC 24 May 2011 

For the forecast initiated at 1200 UTC 24 May 2011 (Figure 33), the 
ellipse formations before recurvature were smaller than the previous forecast sequence. 
However, the largest area of uncertainty for the forecast period was post recurvature 
when the ellipses became elongated along the forecast track. Therefore, the primary 
uncertainty is associated with the timing of the storm as it moved into the mid-latitude 
region. 



Figure 34. The TC forecast track ellipses of each EPS for Typhoon Songda 1200 UTC 24 

May 2011, as defined in Figure 33 
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c. 


Typhoon Songda 1200 UTC 25 May 2011 


By 1200 UTC 25 May 2011 (Figure 33), it was clear that the primary 
uncertainty in the forecast of Songda was in the speed of motion along the track. All 
three EPS exhibited less uncertainty in the short-range forecasts. 



Ensemble Forecasts and 68% Ellipses: Typhoon Songda: 2011052512 
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Figure 35. The TC forecast track ellipses of each EPS for Typhoon Songda 1200 UTC 25 

May 2011, as defined in Figure 33 

2. Typhoon Nanmadol 

On 23 August 2011, Super Typhoon Nanmadol (Figure 31) formed within the 

monsoon trough east of the Philippines and began tracking west-northwestward toward 
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Luzon in a complex steering environment dominated by a subtropieal ridge to the north 
and east (JTWC 2012a). On 25 August, the storm took a poleward turn around the 
steering ridge and rapidly intensified to reaeh super typhoon status by 26 August under 
the favorable environmental influenees of low vertieal wind shear, excellent dual-ehannel 
outflow enhaneed by a Tropieal Upper Tropospherie Trough (TUTT) eell to the 
northeast, and passage over a region of high oeean heat eontent (JTWC 2012a). On 26 
August, Typhoon Nanmadol elipped the northeast tip of Luzon and then moved aeross 
the southern eoast of Taiwan before dissipating in the Taiwan Strait (Figure 31), just 
prior to making landfall in China’s Fujian Province (JTWC 2012a). 

a. Typhoon Nanmadol 1200 UTC 23 August 2011 

Predieted aeross-traek uneertainty for Typhoon Nanmadol was high for all 
three EPS (Figure 36). Overall, the ellipses were elongated in a zonal, or aeross traek 
fashion. The ECMWF and JMA ellipses did not eontain the best-traek positions. 
However, the UKMO had the largest ellipses. Interestingly, the UKMO ensemble-mean 
foreeast traek was closest to that of the aetual TC traek, but had the largest spread. This 
is eonsistent with the XTE summary for the SCSR (Eigure 24) where UKMO had the 
smallest average XTE. 
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Ensemble Forecasts and 68% Ellipses: Typhoon Nanmadol: 2011082312 
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Figure 36. The TC forecast track ellipses of each EPS for Typhoon Nanmadol 1200 UTC 
23 August 2011. Each ellipse signifies a 12 h forecast interval and is colored to 
match the individual EPS as defined in the legend at the top right. The large dot 
inside each ellipse is the corresponding ensemble-mean forecast position. The 

best-track positions are in black 

b. Typhoon Nanmadol 1200 UTC 24 August 2011 

At 1200 EITC 24 August 2011 (Eigure 37), Nanmadol progressed west 
towards Euzon, and the ensemble forecasts begin to shift more towards the best-track 
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positions. The three EPS continue to exhibit uncertainty in across track error. At longer 
forecast intervals the JMA ellipses are becoming parallel to the best-track, which 
indicates uncertainty in the speed of advance. 


Figure 37. 



Ensemble Forecasts and 68% Ellipses: Typhoon Nanmadol: 2011082412 
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The TC forecast track ellipses of each EPS for Typhoon Nanmadol 1200 EITC 
24 August 2011, as defined in Figure 36 


c. Typhoon Nanmadol 1200 UTC 25 August 2011 

At 1200 EITC 25 August 2011 (Figure 38), Nanmadol had progressed 


westward towards Taiwan. The ensemble-mean track of all three EPS continues to 
predict a turn towards Okinawa. At longer forecasts intervals, the UKMO had the largest 
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ellipses, but the ensemble mean foreeast traek was the elosest to the best-traek positions 
in the short- to mid- range. The UKMO ellipses orientation in the long range were turned 
more parallel to the best-track positions indicating it had difficulty accurately forecasting 
the TC’s speed of advance. JMA also showed uncertainty with larger ellipses oriented 
across track in the short-range, and along-track in the long-range speed of advance. The 
ellipses of the ECMWF EPS were the smallest in the short- to mid-range, but continue to 
be shifted to a forecast track that is too far east. 


Ensemble Forecasts and 68% Ellipses: Typhoon Nanmadol: 2011082512 



Eigure 38. The TC forecast track ellipses of each EPS for Typhoon Nanmadol 1200 EITC 

25 August 2011, as defined in Eigure 36 
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3. Summary 

Typhoons Nanmadol and Sondga were seleeted as ease studies beeause of their 
similar initial foreeast tracks, contrasting levels of uncertainty, and impending sortie and 
COR decisions that would have to be recommended by Fleet Oceanographers and made 
by regional, installation, and unit commanders. The decision to sortie or not sortie assets 
is a convincing justification of why probability products are relevant in portraying 
forecast confidence to Fleet Oceanographers and Commanders. 

In the case of Typhoon Nanmadol, the EPS ellipses were elongated normal to the 
forecast and best-tracks, which suggests the potential for across-track errors. Operational 
forecasters who are able to view figures that show ellipse orientation similar to the case 
of Typhoon Nanmadol should be able to comprehend that there is spatial uncertainty in 
the models forecasted track direction. The ellipses in the case of Songda suggested that 
potential along-track errors, which are parallel to the forecasted and best-tracks. This 
case should convey to a forecaster that there is temporal uncertainty in the speed at which 
the storm is forecasted to move along-track. 

Consensus-based forecasts and individual numerical models used to produce 
visual decision aids did not demonstrate positive skill in forecasting the cross-track 
variability that was observed in the case of Typhoon Nanmadol. In the case of Typhoon 
Songda, it did not demonstrate positive skill in forecasting along-track in the RCR and 
MLR. The use of EPS ellipses could have enhanced the representation of uncertainty of 
the forecasts for both cases. 

Compared to the PTE analysis results in the SCSR, the UKMO ellipses for 
Typhoon Nanmadol were the most reliable in capturing the best-track positions. 
However, UKMO ellipses did have lower resolution in the mid- to long-range forecast 
intervals indicating an increased potential for cross-track errors. In the case of Typhoon 
Songda, elongated ellipses in the RCR and MLR show the influence of variable steering 
flow and forecasted TC recurvature influenced by mid-latitude flow patterns. This 
concurs with the ATE statistical analysis for all EPS, which indicated that forecast 
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positions are behind the actual TC tracks, which is observed in the elongated ellipses 
after the forecasted point of recurvature in the RCR and MLR the case of Typhoon 
Songda. 

This case study showed that consensus-based forecasts and individual numerical 
models did not consistently demonstrate value when portraying forecast variability. The 
use of EPS ellipses conveyed potential for high uncertainty over the cross-track and 
along-track directions. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSIONS 

In this thesis, a statistical analysis of uncertainty in TC track forecasts was 
performed by examining the attributes of ensemble-mean forecasts of TC tracks for these 
operational EPS. Thus, the goal was to provide added guidance for improvements to the 
current process of creating operational TC forecasts and visual decision aids. 

The ECMWF ensemble has the highest reliability and resolution of the three EPS 
examined. As discussed, variations in performance of each EPS in individual sub-regions 
indicate that each EPS has unique attributes. The UKMO has the smallest FTE in the 
SCSR, which is an example of how the attributes of individual EPS in specific sub- 
regions could be weighed and exploited to better enhance a deterministic ensemble 
approach to TC forecasting. 

In the case studies, it was evident that consensus-based forecasts and individual 
numerical models did not demonstrate value portraying forecast variability. The use of 
EPS ellipses conveyed potential for high uncertainty over the cross-track and along-track 
directions. 

Based on the results of this study, using individual ensemble predictions systems 
may provide advantages in conveying uncertainty in TC forecasts. The use of the 
multiple single-model ensembles and anisotropic ellipses designed to represent ensemble 
spread does enhance spatial and temporal representation of uncertainty, which 
demonstrates utility with respect to TC track prediction. 

B. RECOMMENDATIONS 

In the analysis section, the use of a Grand Ensemble (GE) of the three EPS in the 
ellipse reliability was not performed. Further research in this subject should also include 
a modification in the MAD analysis using the GPCE circle as the control circle could 
improve the understanding of resolution. 
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Further study into significant biases or systematic errors of along-track and 
across-track errors could reveal sources of uncertainty in speed and space of forecast 
tracks in specific sub-regions. The use of a GE of all three EPS in the WPAC sub- 
regions may also help further define the utility of GE ellipses. Additionally, the sub- 
region boundaries could be improved based on the climatology of storm tracks and storm 
intensities in the WESTPAC basin, while also using a larger period of data availability. 

In addition, the development of forecast uncertainty visual aids using the 
individual EPS performance in the WPAC, and sub-regions, may enhance forecasts and 
visual decision aids produced by JTWC. 

Easily, it would be beneficial to conduct decision analysis based on the cost of 
sortie versus non-sortie and damage vs. non-damage; using contoured product of graded 
probabilities. 
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